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ABSTRACT 



This paper summarizes the development of the Adolescent 
Treatment Outcome Module (ATOM) to monitor the outcomes of mental health 
treatment of adolescents with emotional or behavioral problems . The module is 
intended to: provide reliable and valid information about outcomes of mental 
health care; be applicable across severity levels, clinical settings, 
interventions, and population groups; and be brief and inexpensive to 
administer. The module is composed of seven domains: (1) focal problems; (2) 

diagnostic assessment in five diagnostic categories; (3) multidimensional 
assessment of functional impairment; (4) family burden; (5) acceptability of 
treatment; (6) prognostic or risk-adjustment factors that may influence 
treatment outcome; and (7) assessment of the amount and quality of treatment 
received. (DB) 
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Introduction 

As mental health treatment programs come under closer 
scrutiny by payers who have a range of options for 
therapeutic services, well-designed evaluations of the 
clinical utility, acceptability, and societal'^^alue of services 
are needed (Pfeiffer & Strzelecki, 1990). This paper 
describes development of a module to monitor the outcomes 
of treatment of adolescents with emotional or behavioral 
problems. 

Although an estimated $5_billion is spent. each year in 
providing mental health services to children and adolescents 
(Bums, Taube, & Taube, 1990; Rice, Kelman, Miller, & 
Dunmeyer, 1990), there is virtually no evidence that routine 
mental health care for youth is effective. While some 
isolated studies offer optimistic conclusions about the 
benefits of some forms of treatment (Cause et al., 1994; 
Clark et al., 1994; Henggeler, Melton, & Smith, 1992; 
Scherer, Brondino, Henggeler, Melton, & Hanley, 1994), 
other evaluations suggest that intensive treatment programs 
show minimal benefits compared to standard care (Bickman 
et al., 1995; Heneghan, Horwitz, & Leventhal, 1995). Most 
treatment programs which are considered promising have 
yet to be evaluated, partly because no comprehensive, 
standardized set of instruments have been developed for 
adolescents that are brief, easily administered, and 
inexpensive. To that end, the Adolescent Treatment 
Outcomes Module (ATOM) has been constmcted. The 
objective of the ATOM is to (a) provide reliable and valid 
information about outcomes of care, (b) be applicable across 
severity levels, clinical settings, interventions, and 
population groups, and (c) be brief and inexpensive to 
administer. 



Outcomes modules are sets of standardized, validated 
instruments designed to facilitate the routine and systematic 
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gathering of data on patient response to treatment (Smith, 
Rost, Fischer, Bumam, & Bums, in press). The ATOM is 
the sixth in a series of outcomes modules developed by the 
NIMH Center for Rural Mental Health Services Research, 
University of Arkansas for Medical Sciences. Steps in the 
development of the module, components of the module, and 
preliminary data on a small sample are presented. 

Steps in Module Development 

Module development began with an extensive review of 
diagnostic, prognostic, health service, and methodologic 
issues of adolescent treatment outcome (Robbins & Taylor, 

1 995). A multi-disciplinary panel of experts was convened 
to advise the development team on critical components of 
the module. Experts were recmited from child psychiatric 
epidemiology, functional measurement in children and 
adolescents, child mental health services research, 
stmctured psychiatric diagnostic instmments for children, 
and child psychotherapy research. Based on panel 
recommendations, a draft of the module was composed and 
presented to panel members for evaluation. Recommended 
changes were made, and a pilot study to gather validating 
data has been undertaken. 

Module Components 

The module is composed of seven domains: (a) focal 
problems; (b) diagnostic assessment to establish caseness in 
five diagnostic categories (i.e., anxiety, depression, 
oppositional defiant disorder, conduct disorder, ADD); (c) 
multidimensional assessment of functional impairment; (d) 
family burden; (e) acceptability of treatment; (f) prognostic 
or risk-adjustment factors that may influence treatment 
outcome; and (g) assessment of the amount and quality of 
treatment received. These domains, listed in Table 1 . have 
been identified as most central to understanding outcomes 
of treatment for emotional and behavioral problems 
(Hoagwood, Jensen, Petti, & Bums, 1996). 

Focal Problems 

Youth are referred to clinics not because of a particular 
diagnosis, but because parents, teachers, or others are 
concerned about specific problems they are having at home, 
school, or elsewhere (Weisz & Weiss, 1989). The module 
identifies the primary reasons for initiating treatment, the 
seriousness of these problems, and tracks change in 
presenting problems following treatment. 

Caseness and Symptom Severity 

It is not feasible within the scope of the module to gather 
sufficient data to make diagnoses of disorders using full 
DSM-IV criteria. Therefore, results of item analyses on four 
large data sets were used to generate a much reduced set of 
symptoms that are used to identify likely cases of 
generalized anxiety, separation anxiety, major depression. 
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generalized anxiety, separation anxiety, major depression, 
oppositional defiant disorder, conduct disorder, and 
attention deficit hyperactivity. These symptom items have 
been supplemented with representative symptoms of each 
disorder, selected on the basis of prevalence, to produce a 
measure of symptom severity. 

Functional Impairment 

Therapeutic outcomes in adolescent mental health include 
both resolution of symptoms and enhanced functioning. 
Items designed to address domains of functioning were 
drawn from multiple sources including the Brief Impairment 
Scale designed by Hector Bird (Bird, 1995) from the 
tradition of the Columbia Impairment Scale (CIS: Bird, 
Shaffer, Fisher, Gould, & Staghezza, 1993), and the Child 
Health Questionnaire (CHQ: Kurtin, Landgraf, & Abetz, 
1994). The ATOM also monitors sentinel indicators, 
relatively rare negative events caused directly or indirectly 
by emotional or behavioral problems. Sentinel indicators, 
adapted from draft items of the UNOCCAP study, include 
inability to remain in the home, expulsion or suspension 
from school, and frequent arrests. The following domains of 
functioning are assessed: 

• Functioning in the family. In-home placement is the 
goal of many residential, therapeutic foster care and 
family preservation programs and is the primary 
outcome measure of many evaluations (Gabel & 
Shindledecker, 1 992). Ability of the adolescent to 
remain in the home, relationships with family 
.members, responsibility at home, and ability to follow 
home safety rules are measured. 

• Functioning in school. School-related outcomes 
include acceptance of teacher authority, academic 
progress, and completion of seatwork. Parental 
involvement through frequent calls to come to school 
for behavioral conferences or to pick up an uimily 
child is also included, as are sentinel indicators of 
expulsion and suspension. 

• Community functioning. Successful functioning in 
the community is measured on a sentinel level by 
ability to comply with societal laws. Number of 
contacts with the criminal justice system, arrests, time 
till arrest, incarceration, and residential placement are 
determined. Additionally, measures of involvement in 
community activities and use of leisure time are 
included. 

• Functioning with friends. Peer quality is assessed by 
determining the ability of the teen to make friends 
who do not normally break rules and laws and are not 
regularly in trouble with authority figures. Peer 
relations are addressed by asking both parent and the 
adolescent whether (s)he is able to make friends and 
get along with the friends (s)he has made. 

Family Burden 
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Burden of the family in dealing with the psychopathology of 
an adolescent is measured by the Burden Assessment Scale 
developed by Reinhard and colleagues (Reinhard, Gubman, 
Horwitz, & Minsky, 1994). 

Satisfaction with Care 

Satisfaction with mental health care services is measured by 
the Client Satisfaction Questionnaire (CSQ) modified by 
DeChillo for children (Larsen, Attkisson, Hargreaves, & 
Tuan, 1979). The CSQ is a brief well-validated instrument 
for use with parents. 

Prognostic or Risk-adjustment Factors 

Factors associated with successful outcomes of care were 
identified from the literature and from consultation with the 
expert panel. These factors vary across treatment contexts 
and must be adjusted for in studies comparing outcomes 
across treatment sites. Items include age of onset of 
symptoms, age of onset of aggression, psychiatric and 
substance use comorbidity, mental illness and substance 
abuse history of parents, parental use of mental health 
services, recent family stressors, housing instability, and 
socioeconomic position of the family. Family functioning, 
shown to be a strong determinant of treatment success, is 
measured by the general functioning subscale of the 
McMaster Family Assessment Device (FAD: Byles, Byrne, 
Boyle, & Offord, 1988). 

Treatment Amount and Quality 

There is a growing consensus that mental health treatment 
of adolescents requires multi-modal approaches (Baer & 
Nietzel, 1991). This view is partly based on disappointing 
results from clinical studies of the efficacy of single agents. 
Experience throughout the country with integrated systems 
of care suggests that a combined therapeutic program is 
necessary. Assessment of quality and amount of treatment is 
based on reports of the parent and a chart review guided by 
a checklist of treatment options. Treatment options include: 
(a) medications; (b) parent training; (c) crises services; (d) 
in-home services; (e) individual, family, and group therapy; 
(f) case-management, and (g) school-based treatments. 
Frequency of each intervention is documented. Services 
utilization materials from the National Adolescent and Child 
Treatment Study conducted by the Research and Training 
Center for Children's Mental Health (Silver et al., 1992), 
and the Services Use in Children and Adolescents (Parent 
Self-Report) instrument developed by the task force on 
outcomes research of the American Academy of Child and 
Adolescent Psychiatry were adapted. 

Pilot Test of the ATOM 

Adolescents, age 1 1 to 1 8, from both the inpatient and 
outpatient units at Arkansas Children's Hospital and the 
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outpatient units at Arkansas Children's Hospital and the 
Centers for Youth and Families in Little Rock were 
recruited. Each subject and his/her parent or guardian 
completed the self-report Baseline Assessment of the 
ATOM. The adolescent and parent completed the 
Diagnostic Interview Schedule for Children (DISC: Version 
3.0; Shaffer, Schwab-Stone, Fisher, Cohen, Paicenti, Daves, 
et al., 1993), the Child Behavior Checklist (CBCL: 

Achenbach, 1991) and the Columbia Impairment Scale 
(Bird et al., 1993) to allow us to examine the relationships 
between brief module measures and extensive research 
measures of overlapping constructs. The admitting clinician 
completed the Clinician Baseline Assessment. 

Of the 37 adolescents referred to the research assistant, 3 1 
(84%) agreed to participate in the study. Complete baseline 
data have been collected for all enrolled cases. Subjects 
were 13.8 years old (sd = 1.7) on average, 36% female, and 
39% minority, predominantly African American. Common 
DISC diagnoses include conduct disorder (23%), depression 
or dysthymia (30%) and attention deficit hyperactivity 
disorder (26%). All major outcome measures in the module 
(symptom severity, functioning in the home, functioning in 
the school, functioning in the community, functioning with 
peers) show a distribution approaching normal with no 
evidence of a ceiling or floor effect. We are currently in the 
process of examining correlations between module and gold 
standard measures of comparable constructs, while 
examining patterns in the variation between adolescent and 
parent reports on outcomes. Sentinel indicators show that 
65% of subjects have been suspended from school for one 
day or more; 40% have had contact with the police in the 
past six months; and 15% have made many friends who are 
often in trouble with the authorities. Close to one-fifth of 
parents ( 1 7%) reported missing days at work because of 
their child's problems, and 44% found the household routine 
was upset because of the adolescent's problems. 

On average, the baseline portion of the module, exclusive of 
validating instruments, was completed in 25 minutes (sd = 

8.6) by adolescents, 28 minutes (sd = 7.8) by parents, and 7 
minutes (sd = 9.4) by clinicians. All participants filled out 
the module with a minimum of missing data: (a) < 2% from 
adolescents, (b) < 1% from parents, and (c) < 5% from 
clinicians. Research interviewers began six month follow-up 
evaluations with patients and parents in July, 1996. 

Conclusions 

Routine outcomes monitoring requires instruments that are 
comprehensive, psychometrically sound, and acceptable to 
patients, families, and agencies. They must therefore be 
brief, inexpensive, and administered by non-clinicians with | 
only minimal training. To our knowledge, the prototype | 
ATOM is the only currently available battery of instruments { 
that measures all important outcome domains easily and | 
economically. As such, it holds great promise in advancing | 
routine scientifically rigorous outcomes monitoring in 
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routine scientifically rigorous outcomes monitoring in 
adolescent mental health care. 
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